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1.1 


1.2 


Introduction 


At the ASA Conference in February 2003 there was considerable discussion about 
what we mean by the impact of advice, the extent to which it overlaps with or is 
different from outcomes, and the lack of clear evidence on which we can rely in 
seeking to prove the impact of advice. It was suggested that we conduct a quick 
trawl to see what evidence people had come across. | sent an email to those who 
had been present on the 3 March and a reminder subsequently. 


| am grateful to those who responded, and in particular to James Kenrick, John 
Edwards, Carol Moonlight, and Andy Short. | would also like to thank Penny 
Waterhouse who heard about it and responded, and Carol Buckland from the 
Citizens Advice Kent Probation Project, who provided information about their work. 


The reports | looked at can be easily broken down into categories: national studies 
by Hazel Genn and others; and studies of individual advice agencies/projects. 


Set out below are brief summaries of each report. 


Adam Griffith 
Policy Officer (Legal Services) 


National Studies 


The Effectiveness Of Representation At Tribunals 
By Hazel Genn and Yvette Genn [1989] 


Although primarily concerned with the effectiveness of representation, this study 
necessarily has a great deal to say about the role of advice both because it is the 
necessary precursor to representation, and because many tribunal applicants were 
unrepresented, whether or not they had got advice. 


The study considered 4 tribunals: SSATs, ITs, immigration appeals, and MHRTs. 


It found that: 


e The provision of advice has an important effect on the manner in which cases 
are ultimately decided. [p.63] 


e In SSATs those appellants who had obtained advice were more likely to have 
their cases determined by a hearing, rather than on paper, were more likely to 
attend their tribunal hearing, and more likely to succeed with their appeals when 
they attend. [p.22 —23] 


e Inimmigration appeals, about 80% of appellants had sought advice. [p.29] Those 
who had obtained advice were more likely to have their cases determined by a 
hearing, rather than on paper. [p.35] 


e In ITs about 70% of applicants had sought or received advice. [p.38.] Applicants 
who obtained advice were less likely to withdraw their applications, and more 
likely to settle them. [pp. 44-45, 89-90] The relationship between advice and 
representation varies depending on the source of the advice. [p.48-49] 


e Where information on settlements is available, there are differences between the 
amounts obtained depending on whether advice was obtained, and on the 
source of advice. [p.92] 

e Unlike at SSATs, pre-hearing advice in ITs confers no advantage on 
unrepresented applicants. [p.95] 
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1.3 


Chapter 5 of the report relies on qualitative information from interviews to argue that 
advice to appellants and applicants serves a number of purposes: 


e To avoid incorrect adverse first-line administrative decisions [pp.126-134] This 
reduces the number of occasions in which tribunals are merely performing an 
expensive information-gathering function. [p.244] 


e To filter out of the tribunal system appeals or applications with little or no chance 
of success [pp.134-5] 


e To attempt to settle claims without the need for a tribunal hearing [pp.135-8] This 
is particularly true in ITs, but there is also scope in social security and 
immigration cases. 


e To assist appellants and applicants in the construction of their cases and 
collection of the evidence necessary to prove the case, whether or not they are 
going to be represented. [pp.138-147] 

















1.4 Advice and representation do however affect the time a case takes to come toa 
hearing. In each tribunal, appeals reach a hearing more quickly if the appellant has 
not been advised or is not represented. [p.148] The extent of the delay varies 
between tribunals and varies according to the source of the advice and 
representation. [pp150-54] 

2 Paths To Justice 
By Hazel Genn and others [1999] 

2.1 Chapter 5 considers outcomes in detail, and their correlation to other factors, 
including whether or not the respondent received advice, and the type of advice 
received. It should be noted, firstly, that 70% of those who obtained advice were 
advised that something could be done. [See Figure 5.1, p.147] 

2.2 The analysis is conducted in terms of four possible outcomes: 

e agreement [whether or not court or tribunal proceedings were started] 
e court/ tribunal or Ombudsmen’s decision or order 

e no resolution 

e no action 

2.3 The significance of advice in relation to outcome is first considered in terms of 3 
categories: 

e legal advice [solicitor or occasionally law centre] 
e other advice [all other sources including CABx] 
e no advice. 

2.4 This produced the following correlation between advice sought and resolution of the 

problem: [Figure 5.5, p.154] 
Agreement Court etc No resolution | No action 
order 

Legal 32% 30% 37% 

advice 

Other 33% 11% 55% 

advice 

No advice 37% 7% 44% 12% 
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The extent to which respondents achieved any kind of resolution varied considerably 
depending on the type of problem. Neighbour problems and landlord problems had very low 
resolution rates as compared with divorce and separation problems, tribunal matters, money 
problems and consumer problems. [p.155-6] 


2.5 In some categories of problems it is possible to compare the outcomes for those who 
did and did not seek advice. In some cases, notably neighbour problems and 
landlord problems, the obtaining of legal advice was particularly significant. 


Explaining Outcome 


2.6 A multi variate analysis was used to identify the factors that were significantly 
associated with the outcome of cases. 


2.7 One factor considered was advice received, which was redefined as follows: 


e ‘legal advice’: involving advice from a solicitor [and occasionally law centre] 


e ‘other advice where action taken’: this involved the adviser taking positive steps 
to assist in the resolution of the problem [which | shall refer to as active advice] 


e ‘other advice where no action taken’: this involved the adviser in simply offering 
information or advice or advising the respondent to go elsewhere [which | shall 
refer to as passive advice] 


e ‘no advice’ [p.170] 
Type of advice obtained 
2.8 After controlling for the factors included in the analysis: 
e Cases where no advice was obtained were 45% less likely to be resolved than 
cases where legal advice was obtained 


e Cases where passive advice was received were 30% less likely to be resolved 
than those where legal advice was received. 


e Cases where active advice was received had similar results to those where legal 
advice was received. 


2.9 _ This ‘reflects the discussion in Chapter 3 about the extent to which the provision of 
mere information or advice may be inadequate for certain kinds of cases and certain 
kinds of respondents’. [p.172] 


Conclusion 


2.10 The analysis shows that when legal advice or active advice is obtained, the likelihood 
that a case will end in agreement, adjudication, or court order is increased, so that an 
above average rate of resolution is achieved, whereas the same types of problems 
without legal advice or active advice will have a lower than average rate of 
resolution. [p.178]. 


3 The First LSRC Periodic Survey Of Legal Need 
Pascoe Pleasence and others [2002] 


3.1 The Legal Services Research Centre survey largely follows the pattern of the survey 
carried out by Hazel Genn and others in Paths to Justice. Only limited information is 
presently available. 


3.2 The summary compares the action taken by advisers, i.e. CABx, other advice 
agencies and solicitors in terms of whether they provided active assistance, and the 
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3.3 


4.1 


4.2 


4.3 


4.4 


forms which that took, e.g. contacting or negotiating with the other side, preparing 
paperwork, or attending court. 


The only direct comment about advice and outcomes is as follows: 


‘Advice tended to be sought with more serious problems. As problems grew more 
complex, advice seemed necessary in order to maintain success rates (in terms of 
meeting objectives). Although no significant differences were observed in whether 
respondents obtained their objectives if they had or had not obtained advice, this 
appeared to be a positive reflection of the impact of advice in more serious cases.’ 
[Summary, para 60] 


Studies of Individual Agencies or Projects 


Getting Positive Results 


‘A research report describing user needs, service satisfaction and advice outcomes’ 
in relation to the Camden CAB Service HIV project. 


By Penny Waterhouse [1997] 

The research concluded that: 

e The project was generally successful in meeting needs and getting results, 
particularly in gaining financial security [18 of 20 respondents] 


e The project improved health and well-being: 18 of 20 respondents reported 
health gains, both physical and mental health improvements. [p.2] 


e Gains on housing and employment are more sporadic. Gains on civil rights, 
family and community care, personal support and development are less likely. 


e The CAB makes a significant and positive contribution to outcomes for its users. 
It is likely that these outcomes would not have occurred without CAB 
intervention. [p.3] 


Problems in measuring the extent to which the CAB was responsible for outcomes 
are discussed [p.4] 


In terms of outcomes: 
e 18 of 20 respondents reported health gains 
e 18 of 20 respondents gained some financial security 


e 8of 15 respondents with housing difficulties have resolved these in some 
respect 


e 5of 9 respondents resolved their employment problems, to varying degrees 
e 1 of 9 respondents obtained civil rights successfully. [p.30] 


‘An unexpected outcome is users’ appreciation of the way in which the staff deliver 
the service. The way in which users are treated is an important part of what people 
get from the project; eliciting feelings of self-esteem and positive outlook which is 
reported by users to reduce their stress and improve their physical and mental 
health.’ [p.30] 


An outside professional auditor audited 14 of 20 cases. The advice was generally 
found to be sound. ‘The results of the audit indicate a link between good advice and 
good outcomes: good advice gives good results.’ 
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5.1 


5.2 


5.3 


6.1 


6.2 


6.3 


6.4 


6.5 


6.6 


‘The audit also indicates that there is a difference between user satisfaction with 
advice and outcomes and professional assessment: users are much less critical 
compared to professional evaluation. The value of the service to users is such that 
people trust the project to take care of their interests and assume that any unmet 
needs are due to other people or to the rules.’ [p.37] 


The Effectiveness Of A Local Community Care Rights Project 


A report by Margaret Coombs with Andrew Sedgwick is summarised in the Joseph 
Rowntree Foundation Findings dated August 1988 — Ref 828 


From 1995-97 the Oxfordshire Community Care Advisory Group appointed an 
independent, community care rights worker to help people to obtain their full 
entitlements under the community care legislation. 


A study of the first 180 referrals found that, of the 106 people who worked directly 
with the project, about three-quarters (79) achieved their goals. The findings also 
suggest that better-off, well-organised and assertive people were more likely to be 
successful. 


Details of what was achieved are set out in table 2 of the summary. 


An Evaluation Of The Health And Advice Project: Its Impact 
On The Health Of Those Using The Service 


By S Abbott and L Hobby. July 1999. Report No. 99/63. 
Published by The Health and Community Care Research Unit, University of 
Liverpool. 


This analyses the Health and Advice Project [HAP], a CAB initiative offering benefits 
advice in primary care settings, investigating the impact of the service on the health 
of the users. 


The 48 subjects who received an increase in income after using the HAP service 
believed that the increase had improved their health-related quality of life, and 
showed statistically significant improvements in 3 aspects of health as measured by 
the SF-36: 


e the limits which emotional problems put on the range and extent of all types of 
work [‘role functioning — emotional’] 

e degree of nervousness / calmness, happiness / sadness [‘mental health’] 

e levels of energy and tiredness [‘vitality’] 


The improvements did not continue between 6 and 12 months, although most 12- 
month scores were higher than at first interview. [p.26-27] 


Reductions in GP consultations, prescriptions in general and the prescribing of drugs 
for the first time decreased in the case of those whose income increased, and 
increased in the case of those whose income did not increase. However, these 
changes were not statistically significant. [pp.1, 29] 


The report states that these results represent the first successful attempt to measure 
statistically significant health gain associated with welfare benefits advice services. 


29 of the 48 whose income improved reported improvements in health and/or how 
they felt about life, including: 
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6.8 


6.9 


7.1 


7.2 


8.1 


8.2 


e feeling more optimistic, reduction of stress and worry 

e feeling able to remain at home rather than living in a nursing home 
e better quality of life 

e feeling less depressed and more confident 

e abetter diet 

e the ability to get out more 

e free dental and eye care as a result of receiving benefits [p.22] 


The report refers to other studies: 


One, by Reilly et al [1996] found statistically significant health gain associated with 
general advice services in primary care for a population of homeless people who 
were not registered with a GP at the time of receiving advice. [p.35] 


Appendix 1 discusses the Birmingham CAB project, which began in 1990, and 
involves 7 projects. The service has been evaluated 3 times: 


e to determine the extent of income maximisation achieved 
e a survey of users and others involved 
e seeking to identify the impact on health 


The third study reported significant financial gains, and found a trend towards 
improvement on almost all health —related questions, but the differences were not 
statistically significant. [p.38-39] 


Ten Years Of Casework At Luton Law Centre 
By Adam Griffith [1999] 


This report analyses the results recorded for all cases that were opened and closed 
at the law centre in its first 10 years. It is primarily of interest in terms of its analysis 
of outcomes, and the distribution of the different outcomes as between the main 
areas of work done by the law centre: employment, housing, immigration and 
benefits. 


Appendix | to the report sets out the money obtained for clients in employment cases 
which started in the years 1989 to 1997 inclusive, the amounts of which are broken 
down into different bands. 


Completing The Circle Of Service: Talking To CAB Users 
By Penny Waterhouse [2000] 


This project collected information about the experience of people receiving housing 
advice from CABx. It included six separate projects, one of which considered ‘advice 
outcomes’. Details are given of the main findings in that respect. 


The commentary on the findings includes the following: 


e advice satisfaction — most people are satisfied with the advice they get from the 
CAB, although there is more criticism with advice than with service 
arrangements. There appears to be a link between advice satisfaction and 
expectations: those who get what they expect (in the main, information and 
advice) are more likely to find advice useful; information and advice expectations 
are more likely to be met than expectations for action and advocacy; 
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9.2 


9.3 


e advice outcomes -there is evidence of positive gains as a result of CAB housing 
work. Where problems remain unresolved, this is not seen as the result of CAB 
ineffectiveness. Most people are satisfied by the outcomes from their CAB 
contact, including where this progressed the problem in the right direction. Poor 
results are mainly due to inaction by other agencies. There is evidence that 
sustained action by the CAB is a factor in getting a good result; and there were 
some cases where the survey highlighted further action that the CAB could take 
to improve outcomes. 


e In a number of cases, participating bureaux were more critical than users: about 
the advice they received; about the outcomes; and about the action taken by 
other agencies. Of interest, in one bureau, assessments of outcome made 
without feedback from users were more positive than those made by users. 
There is clearly a difference between satisfaction and effectiveness; and a 
difference between assessments made by different stakeholders. 


The report discusses ‘the problem with outcomes’ and the difficulty of categorising 
them. It suggests that there appear to be three factors that can influence outcomes: 
the capacity of the person to progress the issue; the effectiveness of the advice 
intervention; and the legal and other remedies available. 


Dial House Chester: Independent Assessment Report 
By Dr Leah Pybus [2000] 


This assessment brought together the opinions of 140 clients, staff, café users and 
representatives of external organisations to highlight the strengths of Dial House, 
and to identify weaknesses in its service, and opportunities for improvement in the 
future. [p.45] 


In 1998-99 Dial House responded to 4,325 queries, of which 1,947 concerned 
benefits. It helped disabled people to claim a total of £979,330 in unclaimed benefits. 


[p.2] 
The report states that: 


e ‘For all of those clients using Dial House the support was considered vital to the 
quality of life of clients.’ [p.16] 


e Dial House was central to helping clients overcome feelings of isolation through 
increasing their own awareness of disability issues and enabling clients to obtain 
greater freedom of movement. 


e Dial House support plays a central role in the social and economic empowerment 
of disabled people in the region. [p.19] 


e Feedback from doctors in the area also emphasised that the efforts of Dial 
House certainly contributed to the improvement of the health and well-being of 
patients, and that this was supported by the opinions of patients themselves. 
[p.39] 

e Feedback recognised that the information and advice offered through Dial House 
played an important role in promoting the employment of disabled people by 
local businesses. [p.42] 

e The resources offered through Dial House enable other organisations to provide 
their own clients with a better quality of service, and is of great value at a time 
when support agencies struggle to find the necessary resources to meet the 
needs of clients. [p.44] 
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Mutual Benefits 


A short [1 page] article by Richard Bundy, a principal officer at Salford city council, in 
the Health Service Journal 15.2.01 


The April 1999 IMS Health Survey found that money worries were the most 
frequently cited cause of depression, closely followed by death and illness in the 
family, work worries and relationship problems. 


Over the past few years, advice agencies in different parts of the country have been 
working with GP practices to provide advice in primary health locations, particularly 
on benefit entitlement and debt. 


The article refers to reports on such projects in 


e the West Midlands 

e Derbyshire 

e Wigan 

e Ribble Valley 

e The Broughton area of Salford 


The article describes the Broughton health project. It outlines the financial outcomes 
achieved. It notes that 44% of respondents to a staff questionnaire felt there had 
been a noticeable reduction in the number of subsequent visits from those patients 
referred. 


The Broughton health project is participating in wider research being conducted by 
the Health and Community Care Research Unit at Liverpool University. 


Nacab Kent Probation Project: Report On Recidivism 
Amongst Clients Referred To The Project Between April 1997 
And October 1998 


By Tim Samuel, July 2001 


‘NACAB Kent Probation Project provides a welfare benefits and money advice 
consultancy and casework service to the National Probation Service, Kent Area, staff 
and their clients. 


In a time where “effectiveness” and “what works” is dictating the nature of 
interventions with offenders, we felt it important for us to try and assess whether our 
Project is effective. Social Welfare projects have few concrete ways of determining 
their effectiveness. Records of gains and debts, user satisfaction surveys and equal 
opportunities monitoring are a few of the ways in which effectiveness has been 
measured. However, as the aim of the Probation Service is primarily to reduce 
offending, it was felt an assessment of the impact our assistance has on recidivism 
rates may be an additional measure of effectiveness.’ 


‘The Project allocates a casework category to each client we work with. 


Category A: This work requires very little input from the Project. The work is mostly 
advisory. 


Category B: This is work we have undertaken that has not been completed. This 
happens because either the client has withdrawn from the process or we have been 
unable to continue working with them. 
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Category C: This is short-term casework that has been completed, whether 
successfully or not. The level of the casework is usually single issue. 


Category D: This is in-depth long-term casework. This could be multiple debt work, 
benefit appeals or both.’ 


The report analyses the cases of 301 clients 


The number of clients in each category was 















































Category A 120 

Category B 44 

Category C 87 

Category D 50 

TOTAL 301 

The relevant percentages in terms of re-offending or not were 
Re-offended Did not re-offend 

Category A 46 54 

Category B 43 57 

Category C 34 66 

Category D 20 80 

TOTAL 38 62 














These can be compared with recent statistics suggesting that the percentage of 
offenders commencing community penalties who re-offended within two years 
ranged between 52% and 58%. 


‘For a Social Welfare organisation the above results would seem to compare very 
favourably and represent a vindication of the work done, were it not for the 
complications they create’. 


The report notes that the risk of re-offending can be dependent on gender and age 
as well as the person’s history and type of offences. 


It provides breakdowns of the figures according to gender and age. The figures for 
gender suggest that the project has been particularly successful with women. In 
terms of age however the numbers in each group are too small to enable 
conclusions to be drawn. 


The report concludes: 


‘As has already been noted we do not wish to make any radical claims about the 

effect of Social Welfare advice and assistance on recidivism rates. There are too 
many other variables for us to make such claims. Neither the number of previous 
convictions nor the risk of reoffending has been looked at. 


However what is most pleasing to us is the trend that is visible in all the tables that 
the level of assistance seems to effect the recidivism rate. We would of course hope 
that this is the case but trust that further work can be done to see if there is a link 
between social welfare work and recidivism rates. As the clients we are dealing with 
are also those who are most financially vulnerable, it may also serve to inform any 
debate on the link between poverty and crime.’ 
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Comment 


Further information on these results has been provided by Carol Buckland, the 
project manager. In a telephone conversation on 4.4.03, she said that 


e The Home Office looked at their statistics in more detail and added in the 
likelihood of the person reoffending, based on a risk assessment. Although their 
analysis was less impressive, the draft report did suggest that they had helped 
some specific groups, including women and those with whom they worked in 
depth. 


e The Home Office produced a draft report, but nothing since. The person she has 
been dealing with at the Home Office has moved on. She is unsure whether the 
report will be published. 


e There are a number of factors which affect the likelihood of reoffending, which 
means that it is difficult to show that any improvement was solely due to CAB 
action. 


Get Into The HABIT 


Report on the Pilot Phase of the Project of the Health Advice Benefits Initiative 
Team. 
By Linda Carr and others [2001] 


HABIT is a new service for older people in Merseylive Primary Care Group. It is a 
multi-agency initiative aiming to increase awareness of benefits and services for 
older people and to maximise benefits for those who are eligible. The lead agency is 
Age Concern Liverpool. [p.1] 


The report provides details of the help requested from the project, with particular 
reference to benefits and the need to access services. 


The report suggests a number of ways in which projects such as this can have an 
impact: 


e The report claims that early intervention by HABIT can ease long-term poverty 
and reduce the pressure on the health service and other social services, 
particularly the more acute services. [p.18] 


e The report suggests that claiming benefits can increase the council’s Standard 
Spending Assessment [p.18-19] 


e ‘In addition, given the increasing relevance of income derived from charging for 
community care services, this is a further incentive to ensure people are 
receiving their full entitlement to benefits.’ 


e It also improves revenue collection for the authority — rents and council tax. 


e  Itis also cheaper, quicker and easier for the council to process a claim for 
HB/CTB for a claimant getting IS. 


e Increased spending power benefits the local community as benefits are usually 
spent locally. 


e Research by John Moores University indicates that for every £100,000 spent on 
local shops and businesses, another job is created within the economy. [p.19] 


Further information on the HABIT project is provided by a subsequent report 
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Someone to turn to... . Someone who cares 
An interim summary report, by Lisa Knight [2002] 


This analyses the results achieved for HABIT clients in terms of financial and non- 
financial outcomes. 


It summarises the findings of two focus groups, whose participants explain the 
benefits that they attribute to the project in terms of 


e Increased outcome 
e Improvement in living conditions 
e ‘Psychological’ benefits 


The report summarises the results of a telephone survey, which mentions similar 
benefits and, in addition, the impact of the service on subjective health status. 


Hidden Benefits 
Research into the work of DIAL Waltham Forest, by Benson-Waterhouse [2003] 


This report notes the increasing interest in the question of outcomes. It claims to take 
‘a pragmatic approach’. 


The report analyses the benefits gained by clients in terms of 


e Practical outcomes 

e Health outcomes 

e Quality of life outcomes 
e Personal outcomes 


Details of the findings are set out in Table 1. 
The research found that: 


e DIAL WF has a major impact on the lives of people who use its advice service 


e The effect of increased income on other outcomes, through take-up of disability 
welfare benefits, is marked; in particular in reducing social exclusion 

e Practical and personal gains are substantial and have a positive affect on quality 
of life, including on health 


e Difficulties in achieving good outcomes from social services are reported 
The picture emerging is that: 


e outcomes of value result from advice work and from the way the service is 
provided 


e the direct result of advice work and service delivery are practical and personal 
gains 

e practical and personal gains lead to improvements in health, particularly mental 
health, and to improvements in quality of life. 


The research also highlights some outcomes that are less successful — housing 
adaptations and aids required by those with worsening health; physical health 
outcomes; promoting independence. The reasons for this are not clear from the 
research and DIAL WF will need to explore further the factors that influence these 
findings. 
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